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ABSTRACT 

This paper examines the policy ramifications of 
district-wide portfolio assessment design and implementation in 
Rochester (New York) . A survey of 189 primary grade Rochester 
teachers with respect to their beliefs about, and uses of, portfolio 
assessment examined the link between policy decisions and classroom 
practice. The major finding is that Rochester teachers report that 
portfolios are influencing their assessment practices, but have only 
a limited effect on their curriculum and instructional activities. 

The paper concludes with an analysis of the possible reasons for 
these findings and their policy implications. Appendixes describe the 
developmental stages in early writing and required and optional 
language arts portfolio pieces, and present the survey instrument. 
(Contains 4 tables and 25 references.) (Author/SLD) 
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Abstract 

This paper examines the policy ramifications of district-wide 
portfolio assessment design and implementation in Rochester, New 
York. Through investigation of a survey of 189 primary grade 
Rochester teachers with respect to their beliefs about, and uses 
of, portfolio assessment, and discussion with district policy 
makers, this study examines the link between policy decisions and 
classroom practice. The major finding is that Rochester teachers 
report that portfolios are influencing their assessment practices, 
but have only a limited effect on their curricular and 
instructional activities. The paper concludes with an analysis of 
the possible reasons underlying these findings and their policy 
implications . 
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The Impact of Portfolio Assessment Policy 
on Early Grade Teachers in Rochester, New York 



I . Introduction 

In the quest for better means of evaluating student learning 
and development, assessment has been undergoing a dramatic 
transformation over the past decade. Alternative forms of 
assessment are increasingly being used as companions to, and even 
replacements for, standardized norm- referenced tests. While it is 
generally acknowledged that assessment drives curriculum (O'Connor 
1992, Shepard 1992), there is no explicit connection between 
curriculum and externally mandated standardized tests. Alternative 
assessments are a major departure from standardized tests in that 
they are intended to directly link curriculum, instruction, and 
assessment (Seligman 1989, Resnick and Resnick 1992) . One specific 
form of alternative assessment is portfolio assessment, in which 
samples of student work are collected in folders to detail student 
progress and provide evidence of student abilities (Wiggins, 

1989) . Because portfolio assessment is just beginning to be widely 
used, little empirical evidence exists as to how district 
portfolio policies and implementation strategies are influencing 
teaching practices. 

This paper examines the policy ramifications of decisions 
concerning primary grade district-wide portfolio assessment design 
and implementation in Rochester, New York. Through investigation 
of a survey of 189 primary grade Rochester teachers with respect 
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to their beliefs about, and uses of, portfolio assessment, and 
discussion with district policy makers, this study examines the 
link between policy decisions and classroom practice. For the 
purposes of this analysis, the district-level designers of 
portfolio assessment are referred to as policy makers, while 
administrators and teachers are considered school- level 
implementers . While there are limitations to a self-report 
instrument, this is a useful first-cut look at how implementation 
decisions affect practice. 

The following section provides a review of the relevant 
portfolio assessment literature. Section III describes structure 
of portfolio assessment in Rochester. Section IV relates the data 
collected for this study. Section V details how teachers report 
portfolios are influencing their assessment, curricular, and 
instructional activities. Section VI discusses the findings of the 
study. The article concludes with an analysis of the policy 
implications stemming from the findings. 

II. Review of Existing Research 

Many researchers have theorized about the classroom 
influences of alternative assessments. Wolf, Bixby, Glenn and 
Gardner (1991), for example, expose the "network of activities" 
surrounding the "culture of testing" and suggest that assessments 
should be viewed not as an endpoint, but rather as an "episode of 
learning." More directly, proponents of alternative assessments 
theorize that these assessments can enrich curricula (Livingston, 
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Castle, and Nations, 1989) , enhance instruction (Darling -Hammond 
and Wise 1985) , promote professional development (Bloom 1990) , 
better prepare students with "real world" skills (Resnick and 
Resnick, 1992) , and raise standards (National Commission on 
Testing and Public Policy, 1990) , while still providing 
accountability information (Baron, 1991) . 

Studies which empirically explore the classroom influences of 
portfolio assessment are just beginning to emerge. For example, in 
an examination of a Pittsburgh portfolio project. Wolf (1989) 
describes how students learn to assess their own progress as 
learners. Hebert (1992) reports that portfolios invite students to 
reflect on their work. Tierney, Carter, and Desai, (1991) relate 
how discussions about student work provide teachers with 
opportunities for professional development and shared expectations 
for student performance. Aschenbacher (1994), in a chronicle of 
assistance to teachers implementing portfolio assessment, 
describes the transition of teacher focus from classroom 
activities to student learning goals, the challenges of specifying 
criteria for desired student performance levels, and the realities 
of serious time constraints. The evaluators of the Vermont 
portfolio assessment program, Koretz, Stecher, Klein, and 
McCaffrey (1994) report that the program has had a powerful and 
positive influence on the instruction of many of the state's 
teachers, but is stretching both their time and administrative 
resources . 
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III. Portfolio Assessment in Rochester 1 

Rochester is one of the first cities in the nation to mandate 
portfolio assessment throughout the district. Rochester is an 
urban district serving approximately 9,600 students in 
kindergarten through second grades . The Rochester City School 
District (RCSD) began using portfolio assessment district-wide in 
kindergarten in the 1990-1991 school year. In 1991-1992, they 
began piloting portfolios in the first grade and, in the 1992-1993 
school year they extended portfolio district-wide to the first 
grade as well as piloting them in the second grade. In 1993-1994 
portfolios were mandated in the second grade as well. At the time 
of this research, portfolio assessment in Rochester has been used 
for four years in kindergarten, two years in first grade, and one 
year in second grade. The district plans to continue to roll-out 
portfolio assessment in higher grades as students advance through 
the school system. 

Portfolio assessment was the product of three converging 
movements in Rochester which contributed to a building pressure 
for change of the district's assessment strategies. First was the 
focus, both within Rochester and nationally, on early childhood 
education. With a precipitous decline in the functioning level of 
city pupil, where by 1990, over 60 percent of entering 
kindergarteners had at least one serious problem in language, 
cognition, motor skills, vision, or hearing, RCSD leaders 
concluded that a focused approach on early childhood education was 
essential. Second was an overwhelming kindergarten teacher concern 
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about the effects of mandated standardized testing on young 
children. This resonated with district officials' concerns that 
standardized testing was giving an inaccurate picture of actual 
student competencies and thus hampering their abilities to make 
informed decisions on early childhood education. Third, teachers 
and administrators, observing a rise in student poverty and 
concurrent decline in the functioning level of students during the 
1980s, 2 argued that the district's basal reader approach and other 
"traditional" instructional programs, were not meeting the needs 
of the changing student population. An increased focus on a 
developmental philosophy, of which portfolios are a key component, 
grew out of these factors . 

The implementation of portfolio assessment in Rochester 
consisted of three major activities: first, the development of a 
primary grade educational philosophy and the evolution of 
developmental stages to reflect that philosophy; second, the 
training of teachers in both the concept of portfolio assessment 
and the administrative tasks involved in using portfolio 
assessment, and; third, the quantifying of student performance, as 
represented by the evidence within each portfolio, and the linking 
of this scale to the District's reporting system. 

In 1989, then Assistant Superintendent David Hunt decided to 
replace the end-of-kindergarten standardized test with a form of 
portfolio assessment. A kindergarten portfolio committee of early 
grade Rochester teachers, school administrators, and district 
policy makers was formed to begin discussing the specifics of 
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portfolio assessment. The Committee first agreed upon a set of 
guiding principles for early grade student learning, with an 
emphasis on equity, based upon the notion of differential student 
growth and continuous developmental progress (Rochester Portfolio 
Committee, 1989) . The Committee (and subsequent grade-level 
committees) received RCSD backing, in the form of release time, a 
budget for summer meetings, and administrative support for 
implementation. 

Although there was no formal District mandate for a specific 
educational approach when using portfolio assessment, there was a 
curricular movement away from basal readers towards integrated 
language arts. In developing portfolio assessment, the Portfolio 
Committee adopted a "big tent" philosophy that embraced several 
schools of curriculum thought, including whole language, phonics, 
constructivism, and locally developed curricula. 

Based on its philosophy, the Committee developed a series of 
developmental content standards for student performance. These 
content standards integrated the members ' own experiences with 
national standards recommendations, New York State guidelines, and 
Australian literacy profiles (Rochester Teachers Handbook 1993). 
These content standards spelled out what children should be able 
to do by the end of the second grade in four subject areas: 
language arts, mathematics, science, and social studies. 

Currently, portfolio assessment has only been implemented for 
language arts. Content standards for students are divided into a 
series of "developmental stages, " which are identified as stages A 
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through I. Each stage is a cluster of specific cues, behaviors, 
and skills, that describes a child at that developmental stage. 

The primary grade writing stages, as an example, are shown in 
Appendix A. 

Portfolios are intended to collect concrete evidence of 
children's progression through the developmental stages. Rochester 
teachers are required to collect a series of pieces of student 
work and teacher commentaries that document student performance 
and growth throughout an academic year . These are grade dependent 
but, in general, language arts portfolios are required to include 
samples of student performance (e.g., writing and art samples), 
letter-sound assessments, comprehension assessments, and ongoing 
teacher observational notes. Each document has a minimum frequency 
(e.g., kindergarten students should be asked to draw a self 
portrait three times a year) and teachers are asked to record the 
date a piece was created. Appendix B shows the language arts 
portfolio pieces required in the primary grades . 

The District approached teacher training to implement 
portfolio assessment in three ways. First, the District held 
voluntary summer institutes for teachers to discuss the content 
standards and developmental stages (which were, and continue to 
be, refined) and implementation issues. Second, at the beginning 
of each year that portfolios were mandated at a grade level, the 
District made grade- level city-wide presentations focusing on the 
rationales and fundamental administrative issues of portfolio 
assessment. Third, the District provided release time for in- 
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service training at individual schools for teachers to grapple 
with what it meant in their school to have a common set of 
standards for student work and performance. A cadre of district 
teachers who were involved with the early design and piloting of 
portfolio assessment were also available to discuss portfolio 
implementation issues. 

Portfolio data is now being used by the RCSD in a number of 
ways. First, student portfolio work forms the basis of reporting 
student progress to the community. Beginning in 1993, the District 
changed the way it reported individual student performance to 
parents. Instead of the district's traditional system, 3 the primary 
grade report card reports the developmental stages (with Stage A 
as the lowest stage and Stage I as the highest stage) for language 
arts and math. Additionally, instead of separate report cards for 
each grade level, a single primary report card is now intended to 
show progress from kindergarten through second grade. The District 
standard is for children, by the end of second grade, to reach 
stage E in writing and speaking and listening, and stage F in 
reading. Parents are given a Parent Handbook which details the new 
report card, describes the meaning of the stages, and discusses 
the importance of parent /teacher portfolio conferences. The 
portfolio assessment system requires teachers to hold parent 
conferences twice yearly, focusing on the portfolio contents. 

Second, individual schools use these data for long-range 
school improvement planning (the RCSD is now a school-based 
management district) . Third, portfolio data are used by district 
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leadership for district-wide strategic-planning and decision- 
making. For example, the end-of-the-year developmental stage data 
on all primary students are collected and analyzed by the 
Department of Research, Evaluation, and Testing. The District is 
currently working to formally integrate student performance into 
District promotional policies and hope to use portfolio 
performance for Title I determination. 

IV. Survey Methodology 

The data for this study were collected through a survey of 
primary grade teachers in Rochester in the spring of 1994 . The 
survey was developed in close coordination with the Rochester 
primary grade (PreK-2) Portfolio Committee, the Director of 
Curriculum Development and Support, and the Director of Research 
and Evaluation in Rochester. A Rand Corporation questionnaire 
(Koretz 1992) used previously to survey Vermont teachers about 
their portfolio use provided a framework for this instrument. The 
survey instrument contained 20 Likert-scale questions and six 
open-ended questions. The survey was first piloted with a group of 
20 Rochester teachers and then distributed, through the Rochester 
Portfolio Committee, to the entire primary grade population. A 
decision to make it optional for teachers to put their names on 
the surveys was made to further encourage frankness of response . 
Appendix C contains a copy of the final survey instrument. 

In early May of 1994, the survey was sent to 315 regular 
education kindergarten, first, and second grade teachers by 




12 



The Impact of Portfolio Assessment Policy 12 



mailing it to them at their schools. In late June, an identical 
follow-up survey was sent to each of these teachers at their home 
address. All returned surveys were examined, with particular 
attention to responses to the open-ended questions, in an attempt 
to weed out duplicates; in some cases teachers identified their 
survey as a duplicate. One hundred and eighty-nine teachers 
returned their surveys, for a response rate of 60 percent. 
According to Babbie (1973) this is a respectable response rate for 
a survey of this kind. Of those 189 teachers, 69 placed their 
names on their survey. 

Background Information on Survey Respondents 

The 189 teachers who responded to the survey were fairly 
evenly distributed across the primary grades. Table 1 shows the 
grade levels of the survey respondents. It is a weakness of this 
study that we cannot identify the combination of grades that mixed 
grade level teachers teach. 

These teachers had a wide range of teaching experience . For 
10 of the teachers, or 5.3 percent, the 1993-1994 school year was 
their first year of teaching. Fourteen teachers, or 7.4 percent, 
had 2 to 3 years of teaching experience. Another 14 teachers had 4 
to 5 years of teaching experience. Of the 189 teachers in the 
sample, 23, or 12 percent, had 6 to 9 years of experience. The 
majority of the teachers, 128, or 68 percent of the sample, had 10 
or more years of teaching experience . 
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Table 1 

Grade Levels of Survey Respondents 
(n=189 ) ‘ 



Grade Level 


Number 


Percent 


Kindergarten 


52 


27.51 


First Grade 


69 


36.51 


Second Grade 


56 


29.63 


Mixed Grades 


12 


6.35 


Total 


189 


100.00 



While many teachers who responded to the survey expressed 
reservations about the form of portfolio assessment being used in 
the district, early grade Rochester teachers reported 
overwhelmingly support for the concept of portfolio assessment. 
When asked on the survey "Philosophically, do you support the 
concept of portfolio assessment?" the majority responded in the 
affirmative. Of the 188 teachers who responded to this question, 

72 teachers, or 38 percent, strongly support the concept of 
portfolio assessment; 65 teachers, or an additional 35 percent, 
somewhat support the portfolio concept; 15 teachers, or 8 percent, 
were neutral; 28 teachers, or 15 percent do not support the 
concept of portfolio assessment; while 8 teachers, or 4 percent 
strongly disapprove of the concept of portfolio assessment. There 
were no distinctions in teacher responses by grade levels 



(Kruskal-Wallis =3.51, d.f. =2, p = .173). 
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Issues of General iz ability 

Before statements can be made with any confidence about how 
representative the responding teachers are of the population of 
Rochester teachers, comparisons of demographic characteristics 
between the survey respondents and the rest of the population of 
primary grade Rochester teachers were performed. Because of 
weaknesses in the survey design, we could not compare the surveyed 
teachers directly to the rest of the primary grade teaching 
population in the district. In lieu, we followed a two step 
process to assess generalizability . First, we compared the 
teachers who gave their names on the survey to the rest of the 
primary grade teaching population in the RCSD. Finding no 
differences between these groups, we next compared the teachers 
who gave their names on the survey to the rest of the teachers who 
completed the survey. 

Our first comparison was between those teachers who reported 
their names on the survey (n=69) and all other primary grade 
Rochester teachers (n=246) . Since information on gender, years of 
education, and years of teaching experience was available for all 
the teachers in the district, we were able to compare these two 
groups across these variables. Table 2 shows a multivariate 
analysis of variance (MANOVA) comparing the survey respondents to 



all Rochester teachers. 
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Table 2 



Multivariate Analysis of Variance Comparing Named Survey 
Respondents to All Other Rochester Teachers 

(n=315) 



Variable 


Named Survey 
Respondents Mean 
(n=69) 


All Other Rochester 
Teachers Mean 
(n=246 ) 


F-Value 


Grade 


1.98 


2.14 


1.93 


Gender ( F ) 


.94 


.96 


.73 


Education 


17.48 


17.55 


.19 


Experience 


14.06 


16.59 


3.40- 


l 

V 

A 

O 


*p<.05 **p<. 


01 ***p<.001 





None of the four comparisons between the two groups were 
statistically significantly different at the .05 level, although 
the named survey respondents were slightly less experienced than 
the other Rochester teachers. Therefore, we cannot reject the 
hypothesis that the groups do not represent the same population. 

Because the named survey respondents do not appear different 
than all primary teachers, we moved to our second step of 
conducting a second MANOVA test to determine whether the 120 survey 
respondents who completed the survey anonymously differed from the 
sub-group of the 69 survey respondents who gave their names. Here 
we had less information to go on, since the survey only included 
information on teachers' grade and teaching experience. Table 3 
shows a comparison between the survey respondents and those 
teachers who gave their names. While the grade levels did not 
differ between the two groups, the teachers who responded 
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anonymously to the survey had significantly more experience than 
those who gave their names on the survey. 



Table 3 



Multivariate Analysis of Variance Comparing Named Survey 
Respondents to All Other Survey Respondents 

(n=189) 



Variable 


Named Survey 
Respondents Mean 
(n=69) 


All Other Rochester 
Teachers Who 
Responded to the 
Survey Mean (n=120) 


F-Value 


Grade 


2.07 


2.19 


.77 


Experience 


4.03 


4.45 


5.51* 


l 

V 

A 

M 

O 


*p<.05 **p<. 


.01 ***p<.001 





NOTE: Experience on the survey was on a five point scale. See question 2 
in Appendix C for an explanation of these numbers . 



V. Results From Survey of Rochester Teachers 

This section contains an analysis of teacher responses to 
survey questions. It focuses on three basic educational 
activities: assessment, instruction, and curriculum. Comparisons 
are made both within and across grade levels. The 12 teachers who 
reported teaching mixed grades (six percent of the sample of 189 
teachers) were included in the aggregate data, but were omitted 
from grade level comparisons because it was not known what 
combinations of grades they taught. 
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Influence on assessment practices 

One main purpose of portfolio assessment is to more directly 
assess student skills. Portfolios are designed to contain samples 
of student work which provide authentic evidence of student 
performance (Wiggins, 1989) . In comparison to standardized tests, 
where items are sometimes proxies for student capabilities (for 
example, correct grammar as a measure of effective writing) , 
portfolios are intended to evaluate students' real-life skills 
(Resnick and Resnick 1992) . 

The majority of teachers in Rochester state they are using 
portfolios to assess their students. Figure 1 shows Rochester 
primary grade teacher responses, by grade level, to the statement: 
"I use portfolios to make assessments of student progress." So 
that comparisons can be made between teacher responses across 
grade levels, the bars are scaled according to the percentage of 
respondents giving that answer within each grade level . The 
numbers at the top of each bar represent the number of teachers in 
that grade level who gave that response. For example, in Figure 1, 
three out of 52 kindergarten teachers reported that they never use 
portfolios to assess student progress. These three respondents 
represent approximately six percent of the kindergarten teachers 
who responded to the survey. The discrepancy between the combined 
numbers of the three grade levels and the set of bars depicting 
"all teachers" represents the small number of mixed grade teachers 
who are not detailed in this figure. 
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Figure 1 



Teacher Responses to the Statement: "I use portfolios 
to make assessments of student progress" 
(Percents are by Grade Level Taught) 

(n=189) 



60 % 




Ki ndergarten 
(n=52) 



Fi rst Grade 
(n=69) 



Second Grade 
(n=56) 



All Teacher 3 
(n=1 89) 



Not at All CD Infrequently ^Sometimes ^ Most of the Time HOAlways 



Focusing on the set of bars on the far right of Figure 1, 91 
teachers, or 48 percent, reported that they used portfolios "most 
of the time" or "always" to make assessments of student progress. 
Another 69 teachers, or 37 percent, indicated that they sometimes 
used portfolios to make assessments of student progress. Only 16 
percent indicated that they used portfolios "infrequently" or "not 
at all" for assessing students. 

A comparison of the responses of kindergarten, first, and 
second grade teachers indicates that kindergarten teachers 
reported using portfolios more for assessment of student progress 
than did either first or second grade teachers. This was 
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statistically confirmed with a Kruskal-Wallis (K-W) test (K-W = 
7.99, d.f. = 2, p = .018) . This may be because, at the time of the 
survey, kindergarten teachers had been using portfolios longer 
(for four years) than either first grade (two years) or second 
grade teachers (one year) . There was no statistical difference 
between the responses of the first and second grade teachers 
(Mann-Whitney U = .585, d.f. = 1, p = .444) . 

In explaining how they used portfolios to assess students, 
teachers reported in their open-ended responses that the main 
function that portfolios served is as a means of documenting 
student growth. For one kindergarten teacher this means to "Set 
samples of the child's work side by side and note the areas in 
which the child has improved. " Another typical response was to 
"use beginning, middle, and end of year samples of same tasks to 
compare progress and measure growth. " 

Another indicator of whether teachers are utilizing 
portfolios to assess students is whether teachers are using 
portfolios to help them identify the developmental level of their 
students. The term "developmental level" refers specifically to 
the criteria that the district has constructed to measure student 
progress. Figure 2 shows teacher responses to the statement: "I 

know the developmental level of a child in key areas because of 
portfolio assessment." Because there was no difference between 
grade level responses to this question (K-W = 3.36, d.f. = 2, p= 
.186), responses are reported aggregated across grades. 
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Figure 2 



Teacher Responses to the Statement : " I know the 

developmental level of a child in key areas 
because of portfolio assessment" 

(n=187 ) 




H All Teachers (n= 187) 



Teachers report that portfolios help them identify the 
developmental level of their students . Seventy-one of the 
teachers, or 38 percent, responded that portfolios assist them in 
identifying the developmental level of their children most of the 
time or always . These teachers indicated that this form of 
assessment encourages specificity and greater accuracy in 
assessment. For example, one kindergarten teacher wrote that 
portfolios allow her "to make finer and more accurate observations 
of each child." A second grade teacher indicated that "the 
portfolio is a useful lens by which developmental growth can be 
viewed. " Another teacher stated "My assessments of students are 
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more specific because of the portfolio guidelines." A second grade 
teacher commented that "the letter- sound assessment pin-points the 
exact strengths and needs that a child has in the area of 
phonics . " 

On the other hand, 60 out of the 187 teachers, or 32 percent, 
indicated that portfolios never or infrequently assist them in 
knowing the developmental level of their pupils. These teachers 
generally cited three barriers to the use of portfolios as 
assessment tools. The first was the time involved with portfolios. 
As one typical response stated, "I do not use portfolios as a 
means for assessment. There is not enough time in a day to use a 
portfolio system. Therefore, I choose to TEACH and TEST." 

(Original emphases) . 

The second barrier is that some teachers see no connection 
between portfolio assessment and traditional conceptions of 
academic achievement. As one kindergarten teacher stated, "I have 
to add my own testing of children to really find out where a child 
is at. What is being asked for has little relevance to how a child 
is academically achieving." The third impediment cited by teachers 
is flaws in the Rochester system of portfolio assessment. According 
to one first grade teacher, the "descriptors are too vague and are 
open to a wide range of interpretation by teachers . They do not 
reflect how well the child is doing at that stage." 
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Influence on Instruction 

The literature on alternative forms of assessment suggests 
that these assessments can strongly influence teachers ' 
instructional practices in the classroom. For example, Seligman 
(1989) theorized that alternative assessments clarify 
instructional objectives by focusing teachers on student skills. 
Farr (1990) noted that alternative forms of assessment can 
encourage teachers to adjust instruction to the needs of the 
child. In Rochester, however, teacher survey responses indicate 
that teachers do not perceive that portfolio assessment is having 
a great deal of influence on their instruction. 

Rochester teachers generally do not seem to feel that 
portfolios have influenced their instruction. Figure 3 shows 
teacher responses to the statement: "The use of portfolios has 

changed my instruction. " Only 19 out of the 187 teachers who 
responded to this question, or 10 percent, indicated that 
portfolios have changed their instruction always or most of the 
time. Fifty- six, or 30 percent of the responding teachers, 
indicated that the use of portfolios has "sometimes" influenced 
their instruction. Expanding on this, typical teacher responses 
included "Basically, my instructional program is not dependent on 
the portfolio, " and "My instructional practices have not been 
influenced by portfolio assessment." 
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Figure 3 

Teacher Responses to Statement : 

"The use of portfolios has changed my instruction" 



(N=187 ) 



50 % 




Not at All I nf req ue ntl g So meti mes 



Most of the 
Ti me 



Always 



All Teachers (n=1 89) 



The instructional changes that teachers described as a result 
of portfolio assessment centered upon more individualized and 
small group instruction and more writing activities. According to 
one kindergarten teacher "I feel I know my students much better. . 

. . I'm constantly trying to draw out the original or creative 

responses, looking for the individual kid's uniqueness and/or 
strengths . " Many other teachers mentioned more small group 
instruction. "I have done more small group instruction and/or 
observation, while the rest of the class is busy at various 
centers , " commented one kindergarten teacher . Another teacher 
explained, "Small groups are formed based on student needs 
(remediation and acceleration) from performance tasks. Activities 
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are now built-in (planned) to give me the opportunity to observe 
the descriptors on the stage cards." 

Several teachers mentioned an increase in writing 
instruction. "I do far fewer worksheets - much more writing 
assignments," explained one first grade teacher. There was also a 
group of teachers who changed the structure of their classes, but 
not really their instructional style. One typical response in this 
vein was "I have to have certain requirements in the portfolios so 
my instruction changes to fit them. " 

A few written responses by teachers point to the weakness of 
this particular question as a measure of the influence of 
portfolios on instruction. For example, a few teachers explained 
that portfolios had not changed their instructional practices 
because, as one put it "I feel portfolio assessments have finally 
caught up with good, sound, hands-on early childhood teaching. " 
Other teachers, by contrast, argued that portfolios had a negative 
influence on instruction. According to one kindergarten teacher, 
"My instruction has been influenced because of the time I spend on 
the portfolio pieces and the end of year summary. It [portfolio 
assessment] has been a negative influence on instruction. " 

Therefore, perhaps a better way of examining the influence of 
portfolios on instruction is to find out how frequently teachers 
use portfolios as a teaching tool. Rochester teachers did not 
report using portfolios as teaching tools very often. Figure 4 
depicts the frequencies that teachers reported using portfolios as 
a teaching tool . Responses are aggregated because there were no 
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differences between grade levels in teacher uses of portfolios 
a teaching tool (K-W = 3.28, d.f. = 2, p = .194) . 

Ninety-eight of the responding teachers, or 85 percent, 
reported that they only employed portfolios as a teaching tool 
once a month or less. As one teacher said, "There is too much 
paperwork, which leaves me less time to plan effective use of 
portfolios." Only 29 of the 179 teachers who answered this 
question reported using portfolios as a teaching tool at least 
once a week. As one teacher commented "I use the developmental 
stages as a guide to individualize teaching for students." 

Figure 4 

Frequency of Teacher Use of Portfolios as a Teaching Tool 

(n=179) 
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A final indicator examined in this study of the extent of 
teacher use of portfolios for instructional purposes is how 
frequently teachers talk with students about their portfolios. 
Teachers did not report talking with students very often about the 
work in their portfolios. Figure 5 shows the frequency that 
teachers talk to students about their portfolios. 

Figure 5 

Frequency that teachers talk to their students 
about the work inside their portfolios 

(n=185) 
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Eighty-eight percent of the responding teachers, 162 out of 
185, reported only talking to their students about the work inside 
their portfolios once a month or less. Only 14 of the 185 
teachers, or eight percent, reported talking to students about the 
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work inside their portfolios once a week or more. According to one 
teacher who did, "It has made my teaching more child-centered and 
development ally appropriate. The children now drive what I teach, 
not the basal." Another teacher felt that the students were 
motivated by their portfolios: "The students get excited to see 
their work in the portfolio and are encouraged to see their 
success. They try harder because they want nice work to go in the 
portfolio . " 

A comparison of the responses of kindergarten, first, and 
second grade teachers indicates that kindergarten and second grade 
teachers talk with students about the work in their portfolios 
more than first grade teachers (K-W = 9.32, d.f. = 2, p = .009). 
This may be because kindergarten teachers have had the most time 
to adjust to using portfolio assessment, and second grade teachers 
were trained most recently. First grade teachers, however, did not 
receive training as extensive as their kindergarten counterparts . 
There was no statistically significant difference between the 
kindergarten and second grade teachers (Mann-Whitney U = 3.11, 
d.f. =1, p = .078) . 



Influence on Curricula 

Proponents of alternative assessments state that authentic 
assessment can enrich curricula (Livingston et al . 1989). For 
example, Wolf, LeMahieu, and Eresh (1992) report that classes using 
portfolio assessment have more sustained projects. Resnick and 
Resnick (1992) argue that performance assessments can encourage the 
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replacement of the current "routinized curriculum" with a "thinking 
curriculum" and propose to "place curriculum at the heart of 
assessment decisions" (pages 59-60) . Portfolios have been identified 
as one of the top three curriculum trends by the Association for 
Supervision and Curriculum Development (Vavrus 1990) . 

Rochester teachers, however, did not report that portfolios 
are having a large influence on their curricula. For example, when 
asked if they use student portfolios to develop their lesson 
plans, teachers did not do so with much frequency. Figure 6 shows 
teacher responses to the statement: "I use student portfolios to 

develop my lesson plans . " 



M 



Figure 6 

Teacher Responses to Statement : 

I use Student Portfolios to Develop my Lesson Plans" 
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Ninety-one of the 186 teachers who responded to this 
question, or 49 percent, never or infrequently use portfolios to 
develop their lesson plans. In their written responses, teachers 
expressed uncertainty as to why portfolios would influence 
curricula. As one first grade teacher explained, "I'm not sure 
that portfolios are the type of assessment that would influence 
curricula. Portfolios are more global ... as far as influencing 
daily lessons, I'm not sure that they would do that." Another 
teacher rhetorically asked, "There are many demands and materials 
added to the basic skills to be taught - is this necessary or just 
extra work?" Other teachers, however, described how portfolios had 
become a curricular component. According to one kindergarten 
teacher, "The portfolio has made my teaching more child-centered 
and developmental ly appropriate. The children's work now drives 
what I teach, not the basal." 

Relationship Between Teacher Experience, Support of 
Portfolio Concept, and Survey Responses 

Teacher survey responses about their use of portfolios as 
assessment, curricular, and instructional tools were related to 
both their support of the concept of portfolio assessment and 
their years of teaching experience. Table 4 shows the Spearman 
Correlations between the survey questions about teacher use of 
portfolios for assessment, instruction and curriculum and teacher 
support of the concept of portfolio assessment and teacher 
experience . 




30 



The Impact of Portfolio Assessment Policy 30 



Table 4 

Spearman Correlations Depicting Relationship Between 
Survey Questions and Support of the Concept of 
Portfolio Assessment and Teacher Experience 



Survev Ouestion 


Support of 
Portfolio 
Concent 


Teacher 

Experience 


Support of Concept of Portfolio Assessment 


1.00 


-.34 


Use of Portfolios to Assess Student Progress 


.62 


-.24 


Knowledge of Developmental Level of Child 
Due to Portfolio Assessment 


.55 


-.28 


Change in Instruction Due to Portfolio 
Assessment 


.39 


-.13 


Frequency of Use of Portfolios as a Teaching 
Tool 


.33 


-.09 


Frequency of Discussion with Students about 
the Work Inside Their Portfolios 


.23 


-.11 


Use of Student Portfolios to Develop Lesson 
Plans 


.43 


-.15 



As can be seen from the first column of correlations, 
teachers who support the concept of portfolio assessment tend to 
use portfolios more frequently for assessment, instruction and 
curriculum. The second column indicates the opposite. Teachers 
with more experience tend to use portfolios less for assessment, 
instruction and curriculum. This relationship is compactly summed 
up in the -.34 correlation between teacher support of the concept 
of portfolio assessment and teacher experience; less experienced, 
probably younger, teachers are more likely to support the concept 
of portfolio assessment than are more experienced teachers. 




31 



The Impact of Portfolio Assessment Policy 31 



Summary 

The general pattern that emerged from teacher survey 
responses, in both their Likert-scale responses and written 
comments, is that portfolios are having a strong impact on the 
assessment practices of Rochester primary grade teachers, but only 
a smaller influence on their curricular and instructional 
practices. Before exploring the reasons underneath these findings, 
we should ask to what extent we would expect teachers to change 
their behavior due to portfolio assessment? It is important to 
remember here that the Rochester reform is mandated across all 
primary teachers in the district, a quite different situation from 
a voluntary initiative. One Rochester teacher who reviewed the 
survey instrument suggested that, due to the mandated nature of 
portfolio assessment in Rochester, we should expect all teacher 
responses to be "always." However, the remarkable variation in 
teacher survey responses to most survey questions suggests both 
that teachers were candid in their responses and that portfolio 
assessment, although ubiquitous at the mechanical level, is 
absorbed to varying degrees within the context of school culture. 

VI. Discussion 

Rochester teachers strongly report that portfolios are 
influencing their assessment practices. Since the original 
emphasis of portfolios in Rochester was their role as assessment 
devices, it is encouraging that primary teachers' overwhelmingly 
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report using them to assess children. Rochester administrators 
attribute the different grade-level responses (that kindergarten 
teachers report using portfolios as assessment tools more than do 
first or second grade teachers) to the fact that the school 
district leadership committed far more financial and 
administrative resources to kindergarten, and the fact that 
kindergarten teachers have been using portfolios longer. 

But why are portfolios having a lesser effect on Rochester 
teachers' curricular and instructional activities? There are at 
least three hypotheses that might explain this. The first possible 
explanation is to take the data at face value - that Rochester 
teachers are really not using portfolios as curricular and 
instructional aides. This may be due to the way that the portfolio 
system is constructed in Rochester, with its series of required 
portfolio pieces and suggested completion dates. It is plausible 
for Rochester teachers to fulfill the required portfolio pieces 
without necessarily changing their instructional or curricular 
strategies. As one first grade teacher commented on the survey: 

"My instruction has not been influenced by the portfolios . I 
simply try to keep up with the due dates for the specified 
required pieces and try to give the assessments when I have 
perfect attendance. I use it for evaluation and progress, not 
instruction . " 

By contrast, Vermont’s more flexible portfolio program has 
been reported to have a deeper impact on instruction. In Vermont 
there are no specific pieces of student work that teachers are 
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required to include in student portfolios, but rather they are 
challenged to provide evidence, based on agreed upon standards, of 
student skills and capabilities. Koretz, Stecher, Klein, and 
McCaffrey (1994) report survey data which indicates that the 
Vermont portfolio program has had a strong influence on the 
instructional practices of many teachers. 

A second hypothesis to explain the findings is that 
significant change occurs more slowly over a period of time. 
According to Hall and Loucks (1977) , "change is a process, not an 
event, " which evolves over time from an awareness of a reform to 
actual refocusing of strategies. Under this theory, Rochester 
teachers may initially adopt the mechanics of the innovation and 
only when they are comfortable with the administrative functions 
of portfolio assessment can they begin to adopt it in deeper, more 
complex ways. Since portfolios in Rochester, at the time of the 
survey, have been implemented in second grade for only one year, 
for two years in first grade, and for four years in kindergarten, 
the second hypothesis can be tested. If the hypothesis of teacher 
adjustment is valid, then there should be differences between 
grade levels in collective responses to the survey questions 
concerning the influence of portfolio assessment on assessment, 
curricula, and instructional activities. However, a comparison of 
cumulative responses by grade level indicates there are no 
differences in teacher responses to the questions about 
assessment, instruction, and curricula in the survey (K-W = 3.60, 
d.f. = 2, p = .165). 4 Yet it is still possible, however, that it is 
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too early on the implementation curve for teachers to integrate 
portfolios into their curriculum and instruction. A follow-up 
survey would be able to confirm or reject this theory. 

The third hypothesis is that, because portfolio assessment 
was introduced by the RCSD Department of Research, Evaluation and 
Testing, and initially conceived of as a replacement for 
standardized testing, that teachers do not perceive, and therefore 
do not report, that portfolio assessment is influencing their 
curricular and instructional activities. When we discussed these 
data with both district policy makers and a focus group of 20 
primary grade Rochester teachers, they generally leaned towards 
this explanation of the findings. Many commented that they felt it 
was impossible to use portfolio assessment without having some 
influence on teaching practices. As one first grade teacher 
involved with the design of the portfolio system explained the 
original intent, "If you are going to use the portfolio correctly 
in the classroom, you have to change your teaching practice. You 
can't teach in the 'traditional way' and do the portfolio as 
well . " 

VII. Policy Implications 

Regardless of the explanation, the fact that teachers perceive 
that portfolio assessment is influencing their assessment 
practices, but not their curricular or instructional activities has 
important implications for school administrators and policy makers. 
Policy makers have to conceive, and administrators have to consider 
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how to implement an innovation, such as portfolio assessment, which 
is designed to integrate many of the core components of classroom 
education (assessment, instruction, and curricula) . From this 
research it appears that the signals which emanate from these 
implementation decisions are received by classroom teachers. The 
message from this research is that, if policy makers and 
administrators overly emphasize the assessment component, then 
teachers are likely to respond as did those in Rochester. If 
administrators implement portfolios more as a instructional device, 
then the assessment component may suffer. 

This policy dilemma is further exacerbated by three 
additional complexities. One is the historical practice of 
discussing and implementing curricular, instructional, and 
assessment reforms as if they were unrelated activities for 
classroom teachers. The second is our limited knowledge of how to 
explicitly connect curriculum and instruction to portfolio 
assessment. This area certainly warrants further investigation. 

The third is the particular pressures of the situation where 
short-term expediency may call for emphasis on one component of a 
more complex equation. In the case of Rochester, which has a 
history of changing its early grade instructional programs every 
3-5 years, portfolios were greeted with a certain amount of 
skepticism. Because of the short policy cycle, policy makers saw a 
greater opportunity for immediate administrator and teacher 
acceptance by focusing on assessment reform. As this study shows, 
this approach has longer term consequences . 
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The problem of how to design and implement portfolio 
assessment can be viewed as one issue within a larger set of 
questions involving how to develop closer links between 
curricular, instructional, and assessment activities which are 
traditionally viewed as separate educational operations. If policy 
makers want teachers to engage in serious dialogue over 
professional judgment and student standards which has the 
likelihood of resulting in re-evaluation of the complex 
combination of instructional, curricular, and assessment 
activities which occur in classrooms, then policy makers must 
strategize, and administrators must model, this balanced inter- 
relationship in their design, introduction and roll-out. 
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Appendix A 

Developmental Stages A-E for Writing 

As children master the outcomes, they move through developmental stages. While 
not rigid, the stages are described by behaviors that tend to cluster. 

Children's portfolios should contain concrete evidence of their progression 
through the stages. 

STAGE A 

• Explains marks made on paper with crayons, markers, pencils. 

• Draws pictures and names what is drawn (even if recognizable only to child) . 

• Combines drawing and "writing." 

• Shows interest in letters. 

STAGE B 

• Repeats the names of letters as teacher writes them. 

• Recognizes letters in own name independently. 

• Recognizes that drawing is different from writing. 

• Uses letters or letter-like forms randomly to convey a message; is able to 

"read" the message. 

• Copies words from the environment. 

STAGE C 

• Writes in a horizontal f ashion--lef t to right, top to bottom. 

• Writes legibly using spaces between words. 

• Shows evidence of initial awareness of beginning and some ending sounds. 

• Uses invented spelling; uses more than one letter sound to a word. 

• Investigates spellings of words. 

• Writes words in a logical order to make a sentence that can be read. 

STAGE D 

• Reads own written work to self and/or others; reflects on own written work. 

• Usually keeps to the topic when writing stories that have a beginning, middle, 
and an end. 

• Produces independent pieces of writing some of the time. 

• Spells accurately words used most frequently and consistently makes informed 
attempts to spell unknown words. 

• Begins to experiment with writing for different purposes, different audiences. 

• Writes about feelings and direct experiences. 

• Uses reality and imagination to create characters/situations when writing. 

• Begins to support own opinions in writing. 

STAGE E 

• Produces independent pieces of writing using the conventions of English 

(spelling, capitalization, end punctuation, quotation marks) most of time. 

• Develops ideas into paragraphs . 

• Writes effectively on a variety of topics with audience, purpose in mind. 

• Writes in a variety of genres - -poems , journals, letters, invitations. 

• Makes appropriate shifts from first to third person in writing, sometimes 
experimenting with dialogue. 

• Edits work to the point where others can read it. 

• Begins to use dictionary, thesaurus, word checker to assist in vocabulary and 
spelling . 

• Summarizes the essential facts and ideas of a text in writing. 

• Expresses an opinion with convincing support. 
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Appendix B 

Required and Optional Language Arts Portfolio Pieces Primary Grades 



Kindergarten 

Required Pieces 

Kindergarten Screening Report lx/year 

Self-Portrait 3x/year 

Writing (scribbles, copied words, dictated 

and copied story, word lists, etc.) Ongoing 

Printing of name 2x/year 

Art (line drawing, child's choice of art, etc.) lx/year 

Written numbers and letters 2x/year 

Art depiction of a story lx/year 

Developmental stage designations Ongoing 

Anecdotal Observations 2x/year 

Summary of Observations June 

Language arts reporting form June 

Optional Pieces 



Self-evaluation, Reading logs. Photographs, Documentation 

per teacher choice, Summary of major parent conferences 



First Grade 

Required Pieces 

Letter/sound assessment 3x/year 

Comprehension assessment lx/year 

Writing samples 3x/year 

art/self-portrait 2x/year 

Developmental stage designations Ongoing 

Anecdotal observation form 2x/year 

Anecdotal summary sheet June 

Language arts reporting form June 

Optional Pieces 



Self-evaluation, Reading logs. Photographs, Documentation 

per teacher choice, Summary of major parent conferences, 
Reading survey, Oral reading summary 



Second Grade 

Required Pieces 

Letter/sound assessment 3x/year 

Independent writing assesment 3x/year 

Comprehension assessment 2x/year 

Art/self-portrait 2x/year 

Developmental stage designations Ongoing 

Self-evaluation 2x/year 

Reading log Ongoing 

Anecdotal Observations 2x/year 

Summary of Observations June 

Language arts reporting form June 

Optional Pieces 



Photographs, Documentation per teacher choice, 
Oral reading summary 
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Appendix C 
Survey Instrument 
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End Notes 



1 The information for this section comes from interviews with Dr. Jean 
Slattery, Rochester's Supervising Director of Curriculum Development and 
Support, Ann Pinnella-Brown, the Coordinating Director Research, Evaluation, 
and Testing, and Project Administrator Andrew MacGowan, during October, 1994, 
as well as examination of district documents. 

2 In 1980, fewer than one quarter of students in Rochester were eligible for 
free and reduced lunch. Today, over 75% of students are eligible for 
free/reduced lunch. Additionally, by the late 1980s approximately 50% of the 
kindergarten students had serious deficiencies in the areas of language, 
cognition, motor skills, vision, and/or hearing. 

3 Based on a three stage scale of satisfactory, needs to improve, and 
unsatisfactory . 

^ This analysis was done by adding up teacher responses to survey questions on 
assessment (questions 13 and 14), instruction (questions 12, 18, and 19), and 
curricula (question 10) . Totals ranged from 7 to 30. This range was then 
reduced to three categories (low, medium, and high) by examining the 
distribution of responses. 
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